In order to realize safe driving or automatic driving of a vehicle, a technique to detect an obstacle is necessary. This technique is classified into a utilization of a laser or a supersonic wave, and a utilization of a TV camera. An apparatus using the laser is expensive. In an apparatus using the supersonic wave, resolution of the supersonic wave is low and detection accuracy of the obstacle is correspondingly low. Furthermore, an active sensor using the laser or the supersonic wave cannot recognize a driving lane of the vehicle. On the contrary, TV cameras are relatively inexpensive and have suitable resolution, instrumentation accuracy, and instrumentation limit. Furthermore, the driving lane is also recognized. In case of utilizing a TV camera, a method using one TV camera and a method using a plurality of TV cameras (stereo camera) are selectively used. In the method using one TV camera, a road area and an obstacle area are separated from one image taken by the one TV camera according to intensity, color, or texture information. For example, an intensity area of the average level whose brightness degree is low, i.e., a gray area, is extracted from the image as the road area. An area not including texture may be extracted from the image as the road area. The other area in the image is regarded as the obstacle area. However, many obstacles include intensity, color, or texture similar to the road. Accordingly, in this method, it is difficult to separate the obstacle area from the road area.
On the other hand, in the method using a plurality of cameras, the obstacle is detected based on three-dimensional information. In general, this method is called “stereo view”. For example, two cameras are located at the left side and the right side; the same point in three-dimensional space is corresponded between the left image and the right image; and a three-dimensional position of the point is calculated by triangulation. If a position and a posture of each camera for a road plane are previously calculated, the height of an arbitrary point from the road plane in the image is calculated by the stereo view. In this case, the obstacle area is separated from the road area by the height. In the stereo view, a problem in case of utilizing one camera is avoided.
However, in regular stereo view, a problem such as search of correspondence points is included. In general, the stereo view is a technique to calculate a three-dimensional position of arbitrary point of the image in coordinate system fixed to the stereo camera (Hereinafter, it is called “stereo camera coordinate system”). The search for correspondence points represents search calculation necessary for correspondence of the same point in space between the left image and the right image. The calculation burden is extremely high. The search for correspondence points is a factor preventing realization of the stereo view.
In this case, if the road area and the obstacle area are simply separated in the image, the search for correspondence points is not always necessary. For example, the height of an area from the road plane is decided as follows. Assume that a projection point of the same point on the road plane to the left image and the right image is respectively (u, v), (u′, v′). The following equation (1) is concluded.
                                          u            ′                    =                                                    h                ⁢                                                                  ⁢                11                ⁢                u                            +                              h                ⁢                                                                  ⁢                12                ⁢                v                            +                              h                ⁢                                                                  ⁢                13                                                                    h                ⁢                                                                  ⁢                31                ⁢                u                            +                              h                ⁢                                                                  ⁢                32                ⁢                v                            +                              h                ⁢                                                                  ⁢                33                                                    ,                                  ⁢                              v            ′                    =                                                    h                ⁢                                                                  ⁢                21                ⁢                u                            +                              h                ⁢                                                                  ⁢                22                ⁢                v                            +                              h                ⁢                                                                  ⁢                23                                                                    h                ⁢                                                                  ⁢                31                ⁢                u                            +                              h                ⁢                                                                  ⁢                32                ⁢                v                            +                              h                ⁢                                                                  ⁢                33                                                                        (        1        )            
{right arrow over (h)}=(h11, h12, h13, h21, h22, h23, h31, h32, h33): parameter of the camera dependent on a position and a posture on the road plane, a focal distance of lens of the camera, and the origin of the image.
“{right arrow over (h)}” is previously calculated by projection points (ui, vi), (ui′, vi′) (i=1, 2, . . . , N) of at least four points on the road plane to the left image and the right image. By using the equation (1), in case that arbitrary point P (u, v) on the left image exists on the road plane, a correspondence point P′ (u′, v′) on the right image is calculated. If the point P exists on the road plane, the points P and P′ are a correct pair of correspondence points and intensities of the two points are equal. Accordingly, if an intensity of the point P is not equal to an intensity of the point P′, the point P is decided to belong to the obstacle. In this method, the height of the arbitrary point in the image from the road plane is directly decided using the equation (1). Coefficients in the equation (1) are calculated by projection points of at least four feature points on the road to the left and right images. Accordingly, the search for correspondence points between the left and right images is not necessary.
If the vehicle is moving at low speed on a flat floor in an indoor environment, {right arrow over (h)} is fixed. Therefore, the obstacle is correctly detected using {right arrow over (h)} calculated once. However, if the vehicle is moving outdoors, relative posture of each camera on the road plane changes regularly because of vibration of the vehicle itself and changes in the inclination of the road. As a result, the parameter {right arrow over (h)} of the camera changes in proportion to the movement of the vehicle. Therefore, if the parameter {right arrow over (h)} calculated during a still moment of the vehicle is utilized, if the camera image is transformed by the equation (1), and if an obstacle is detected by subtracting the transformed image from another camera image, the detection accuracy extremely falls.
As mentioned-above, the obstacle detection apparatus is classified into a type using a laser or a supersonic wave (active), and another type using TV camera (passive). However, the obstacle detection apparatus using the laser or the supersonic wave is expensive, the instrumentation accuracy is low and the driving lane is not recognized. Furthermore, in the obstacle detection apparatus using a TV camera, a usable environment is restricted, the search for correspondence points whose calculation load is high is necessary, and the vibration of the vehicle while moving and the inclination of the road are not taken into consideration. As a result, the ability extremely falls in the outdoor environment.